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INTRODUCTION 


The  accompany log  Evaluation  Instrument  contains  487  plan  factors  which 
have  been  identified  as  being  of  varying  degrees  of  importance  to  the  opera¬ 
tional  capability  of  a  community  fallout  shelter  system.  These  factors  are 
organized  into  33  subject-matter  categories.  The  categories  are: 


GENERAL 

OPERATIONAL  PLAN 

PLAN/COMMUNITY  COMPATIBILITY 

POPULATION  INDOCTRINATION  AND  TRAINING 

SHELTER  ASSIGNMENTS 

SHELTER  MANAGEMENT  (Pre -Emergency) 

SHELTER  STOCKING 

SHELTER  DESIGN 

SHELTER  UTILIZATION  PLAN 

PERIODIC  MAINTENANCE 

POST-SHELTER  PLANNING  (Pre -Emergency ) 

COORDINATION 

WARNING  SYSTEM 

INGRESS 

RADIOLOGICAL  DEFENSE 

SHELTER  MANAGEMENT  (in-Shelter) 

ATMOSPHERE  CONTROL 


WATER 

FOOD 

SLEEP 

SANITATION 

MEDICAL 

SPACE -VOLUME  REQUIREMENTS 

LIGHTING 

POWER  SUPPLY 

CONTINGENCY  PLANNING 

COMMUNICATION 

CONTROL 

MAINTENANCE 

NOISE 

TRAINING 

RECREATION  AND  RELIGIOUS  ACTIVITIES 
POST -SHELTER  PLANNING  (in -Shelter) 


Grouping  the  plan  factors  by  subject-matter  categories  (and  in  some 
cases  sub -categories) serves  the  purpose  of  providing  a  classification 
system  whereby  the  user  can  readily  locate  a  particular  plan  factor.  It  also 
provides  a  basis  for  summarizing  the  results  of  an  evaluation  in  terms  of 
major  functions  of  the  system  such  as  Shelter  Management  and  Radiological 
Defense. 

Each  plan  factor  item  in  the  Evaluation  Instrument  has  been  assigned  a 
weighted  score  corresponding  to  its  Judged  importance  to  system  effectiveness. 
This  score  is  based  on  ratings  of  the  effect  of  not  providing  for  the  factor 
in  a  comaunity  shelter  system.  Be  prepared,  then,  to  think  of  the  weighted 
scores  as  being  negative  in  value.  Possible  item  weights  range  from  1  to  100. 
The  assigned  weight  is  directly  proportional  to  the  Judged  seriousness  of 
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omission  of  the  factor  from  the  shelter  system. 

The  Evaluation  Instrument  is  intended  for  use  by  Office  of  Civil  Defense 
personnel  whose  responsibility  it  is  to  assess  the  operational  capability  of 
community  fallout  shelter  systems  by  examination  of  descriptions  submitted  to 
them  for  this  purpose.  It  is  designed  to  be  applicable  in  evaluating  systems 
which  may  vary  across  a  wide  range  in  parameters  such  as  community  size  and 
shelter  types.  It  can  also  be  used  to  evaluate  shelter  systems  in  all  stages 
of  development  from  a  completely  "paper"  system  to  one  which  is  fully  opera¬ 
tional.  Appropriate  shifts  in  verb  tenses  within  individual  items  are  all 
that  is  required  to  make  the  instrument  applicable  regardless  of  the  phase  of 
system  development. 

Scores  are  obtained  for  each  category  based  on  the  presence  or  absence 
of  provisions  for  applicable  plan  factors.  A  profile  of  category  scores  is 
prepared  to  illustrate  the  results  of  the  evaluation.  Weak  points  in  system 
planning  are  critiqued  to  provide  guidance  for  local  civil  defense  officials. 
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EVALUATION  PROCEDURE 


Prior  to  beginning  the  item-by-ltem  evaluation  of  a  shelter  system,  two 
preparatory  steps  should  be  taken.  First,  read  all  the  material  pertinent  to 
the  system  until  you  are  thoroughly  familiar  with  its  arrangement  and  the 
method  of  classifying  information.  This  will  be  time  well  spent  particularly 
if  the  material  cannot  be  rearranged  to  parallel  the  Evaluation  Instrument. 
Second,  if  possible  and  necessary,  rearrange  the  material  to  parallel  the 
Evaluation  Instrument.  Accomplishing  these  two  things  first  will  greatly 
facilitate  the  evaluation  and  reduce  the  possibility  of  under -evaluating  a 
shelter  system  by  overlooking  information  pertaining  to  the  plan  factors. 

Upon  completion  of  the  preparatory  phase,  carry  out  the  following  four 

steps: 

1.  READ,  IN  TURN,  EACH  PLAN  FACTOR  ITEM 

2.  DETERMINE  WHETHER  OR  NOT  THE  FACTOR  HAS  BEEN  PROVIDED  FOR 

3.  MARK  THE  EVALUATION  BOOKLET  TO  INDICATE  WHETHER  OR  NOT 
THE  PLAN  FACTOR  HAS  BEEN  PROVIDED  FOR 

k.  CHECK  THAT  EACH  ITEM  HAS  BEEN  MARKED 

The  following  detailed  procedures  will  aid  in  carrying  out  these  four 

steps : 

1.  READ  THE  PLAN  FACTOR  ITEM 

With  the  exception  of  the  items  in  the  first  category,  all  verbs  are 
future  tense.  This  enables  you  to  evaluate  plans  to  provide  for  the  various 
plan  factors  although  no  other  action  has  been  taken.  When  the  shelter  system 
is  operational  or  becoming  so,  simply  read  the  item  in  the  present  tense,  e.g., 
"are  ...  to  be"  would  be  read  "are  .  .  .  being"  or  "are." 

2.  DETERMINE  WHETHER  OR  NOT  THE  FACTOR  HAS  BEEN  PROVIDED  FOR 

The  ease  with  which  this  step  can  be  performed  will  depend  a  great  deal 
on  the  format  and  comprehensiveness  of  the  material  being  evaluated.  If  the 
system  of  classifying  the  data  approximates  that  used  in  the  Evaluation 
Instrument,  this  task  will  be  a  relatively  simple  one.  If  not,  the  difficulty 
in  positively  determining  the  presence  or  absence  of  a  plan  factor  will  be 
directly  proportional  to  the  disparity  in  organization  of  the  material  and  the 
Evaluation  Instrument.  A  good  working  knowledge  of  where  in  the  material 
various  aspects  of  the  system  are  treated  will  be  most  helpful  at  this  point. 

In  most  cases,  it  will  be  a  matter  of  determining  whether  something  has 
been  done  or  will  be  done  to  provide  for  a  given  plan  factor  as  opposed  to  no 
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mention  of  it  in  the  material.  Although  adequacy  of  provision  is  a  very 
important  aspect,  little  or  no  objective  data  are  available  at  this  time  for 
use  in  making  such  decisions  on  most  of  the  plan  factors.  Where  such  data 
are  available,  they  are  included  directly  below  the  pertinent  plan  factor  item 
and  identified  by  the  word  STANDARD.  The  level  indicated  in  the  standard  is 
considered  minimal  in  providing  for  the  factor. 

The  decision  as  to  whether  a  given  plan  factor  has  or  ha9  not  been  pro¬ 
vided  for  should  be  tempered  by  several  considerations.  Among  these  are: 

(a)  What  stage  of  development  is  the  system  in  (preliminary 
planning,  detailed  planning,  developmental,  operational)? 

(b)  Are  all  aspects  of  the  system  discussed  to  the  same  level 
of  detail? 

(c)  Have  assumptions  been  made  regarding  basic  system  objectives 
which  preclude  the  necessity  of  providing  for  the  factor? 

(d)  Have  other  plan  factors  been  provided  for  in  such  a  manner 
that  the  factor  under  consideration  is  also  provided  for? 

Try  to  be  as  objective  as  possible  in  determining  the  presence  or  absence 
of  a  plan  factor.  However,  if  you  must  be  subjective  in  reaching  a  decision, 
be  consistent  in  the  manner  in  which  you  exercise  your  Judgment.  For  example, 
if  you  infer  provision  for  a  plan  factor  in  one  instance,  given  the  same  amount 
of  information,  you  should  make  like  inferences  for  other  factors.  In  the 
interest  of  providing  a  fair  and  helpful  evaluation,  however,  a  plan  factor 
should  not  be  considered  as  having  been  provided  for  without  definite  state¬ 
ments  and/or  data  to  that  effect  in  the  system  documentation. 

3.  MARK  THE  EVALUATION  INSTRUMENT  TO  INDICATE  WHETHER  OR  NOT  THE  PLAN  FACTOR 
HAS  BEEN  PROVIDED  FOR 

Depending  on  the  outcome  of  the  preceding  step,  there  are  three  possible 
ways  to  mark  the  item.  The  three  possible  item  responses  are  listed  on  the 
right  hand  side  of  the  page  opposite  each  item.  They  are:  N/A  for  "not 
applicable";  Y  for  "yes";  and  a  number  which  is  the  item  weight  for  "no." 

If  you  determine  that  the  plan  factor  is  not  applicable  to  the  shelter 
system  being  evaluated,  circle  the  N/A  to  the  right  of  the  factor. 

Example : 

On  an  above-ground  shelter,  are  barriers  to  be  used  to 

reduce  blast  effects?  f 

If  you  determine  that  the  plan  factor  has  been  provided  for,  circle  the 
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Y  to  the  right  of  the  factor. 

Example : 

Is  all  primary  shelter  space  to  have  a  protection 
factor  of  100  or  better? 

If  you  determine  that  the  plan  factor  has  not  been  provided  for,  circle 
the  number  to  the  right  of  the  factor. 

Example : 

Are  provisions  to  be  made  for  stocking  food  in  the 
shelters?  n/A  Y 

The  item  weight  is  used  rather  than  "no"  for  a  negative  response  since 
"no"  answers  are  the  only  ones  to  be  given  a  numerical  value  in  the  scoring 
system.  This  greatly  simplifies  the  scoring  procedure  as  will  be  explained 
in  the  next  section. 

h.  CHECK  THAT  EACH  ITEM  HAS  BEEN  MARKED 

Before  leaving  a  category  and  proceeding  to  the  next,  check  each  item  to 
make  sure  one  of  the  three  possible  response  choices  has  been  circled.  One 
of  the  three  possible  responses  must  be  circled  for  each  item.  In  the  case  of 
control  items  (see  next  paragraph),  check  that  the  instructions  concerning  them 
have  been  followed. 


CONTROL  ITEMS 

Several  of  the  categories  contain  items  which  determine  the  response  to , 
several  or  all  items  following  them  in  the  category  when  they  are  responded  to 
negatively.  These  items  are  followed  by  notations  in  capital  letters  which 
identify  the  control  item  and  the  items  which  are  affected  by  it.  In  all  case^  , 
when  a  control  item  receives  a  negative  response  those  items  which  are  affected 
must  be  marked  N/A. 
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SCORING  PROCEDURE 


Scoring  the  Evaluation  Instrument  involves  the  following  four  steps: 

1.  COMPUTE  TOTAL  SCORE  FOR  EACH  CATEGORY 

2.  TRANSFER  CATEGORY  TOTALS  TO  COMPUTATION  SHEET 

3.  COMPUTE  CONVERTED  SCORES  FOR  EACH  CATEGORY 

k.  PLOT  CONVERTED  SCORES  ON  OPERATIONAL  CAPABILITY  PROFILE 

A  more  detailed  description  of  how  to  perform  these  four  tasks  is  given 

below: 

1.  COMPUTE  TOTAL  SCORE  FOR  EACH  CATEGORY 

Computing  the  total  score  for  a  category  is  simply  a  matter  of  summing 
those  numbers  in  the  third  column  of  the  response  choices  which  have  been 
circled.  Do  this  carefully  and  double  check  your  results.  It  is  easy  to 
make  errors  of  inclusion  or  exclusion  when  adding  only  certain  portions  of 
a  column  of  numbers . 

Enter  the  results  of  your  addition  in  the  space  provided  to  the  right  of 
the  words  CATEGORY  TOTAL. 

When  the  items  for  a  single  category  occupy  more  than  one  page,  additional 
summing  of  sub -totals  will  be  required.  Each  page  of  a  multi -page  category 
will  have  a  sub-total  entry  at  the  bottom  entitled  Page  Total  with  an  appropri¬ 
ate  space  to  the  right  of  it  for  recording  the  sum  of  the  encircled  numbers 
for  that  page.  The  last  page  of  a  multi -page  category  will  have  two  types  of 
sub -total  entries  plus  the  CATEGORY  TOTAL  entry.  The  two  sub -total  entries  are 

a  "Page  Total"  for  the  last  page  and  a  "Total  for  Page  _ "  for  every  page 

in  the  category. 

To  arrive  at  a  category  total  for  a  multi -page  category,  simply  carry  the 
Page  Totals  forward  to  the  last  page  of  the  category  and  enter  them  in  the  space 
provided.  Although  the  order  in  which  sub -totals  are  recorded  will  not  affect 
the  CATEGORY  TOTAL,  a  specific  space  will  be  provided  for  each  page  total  for 
cross-check  purposes.  For  example,  the  sub-total  and  category  total  entries  on 
the  last  page  of  a  category  three  pages  long  would  appear  like  this: 

N/A  Y  _ 

Page  Total  ^  ^ 

Total  for  Page  18  3 

Total  for  Page  19  'IQ 

Total  for  Page  20  / 3 

CATEGORY  TOTAL  £ 
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TRANSFER  CATEGORY  TOTALS  TO  COMPUTATION  SHEET 

The  COMPUTATION  SHEET  near  the  front  of  the  Evaluation  Instrument  booklet 


has  two  major  sub-divisions.  The  plan  factor  categories  are  listed  on  the 
left  side  of  the  page  in  the  order  they  appear  in  the  Evaluation  Instrument. 
The  right  side  of  the  page  contains  a  Maximum  Score  and  spaces  for  entering 
the  Category  Total  score  and  Converted  Score  for  each  category. 

Column  headings  across  the  upper  right  hand  side  of  the  COMPUTATION 
SHEET  identify  the  spaces  to  be  used  l'or  the  various  scores.  The  columns  are 
arranged  to  facilitate  the  computation  of  the  Converted  Scores  for  each  cat¬ 
egory. 

Place  the  Category  Total  scores  calculated  in  step  1  in  the  spaces  pro¬ 
vided  under  the  column  heading  Category  Total  across  from  the  correct  category 
titles. 

3.  COMPUTE  CONVERTED  SCORES  FOR  EACH  CATEGORY 

As  can  be  seen  on  the  Computation  Sheet,  each  category  has  the  same 
Maximum  Score.  To  compute  the  Converted  Score  for  a  category  simply  sub- 
trace  the  Category  Total  score  from  the  Maximum  Score. 

If  important  plan  factors  within  a  category  are  not  provided  for  in  the 
shelter  system  it_  is  quite  possible  for  the  Converted  Score  to  be  zero  or 
even  negative.  When  the  Categc.;'  Total  score  is  larger  than  the  Maximum 
Score  be  sure  to  include  a  minus  3?gn  in  front  of  the  Converted  Score. 

This  seemingly  impossible  situation  of  being  able  to  achieve  a  Category 
Total  score  which  is  larger  than  the  Maximum  Score  is  a  consequence  of  the 
scoring  technique.  A  full  explanation  of  the  development  of  the  scoring 
technique  and  the  rationale  for  it  can  be  found  in  the  Final  Report  covering 
the  technics'1  aspects  of  the  development  of  the  Evaluation  Instrument. 

U.  PLOT  CONVERTED  SCORES  ON  OPERATIONAL  CAPABILITY  PROFILE 

A  profile  chart  for  plotting  the  Converted  Scores  is  provided  in  the 
front  of  each  Evaluation  Instrument  booklet.  When  completed,  the  chart 
provides  a  graphic  presentation  of  the  shelter  system's  operational  capability. 
A  sample  profile  chart  is  included  on  the  next  page  to  illustrate  this. 

The  profile  format  resembles  a  bar  graph.  The  area  assigned  to  each 
category  is  identified  by  the  category  title  at  the  top  of  the  chart.  Posi¬ 
tive  Converted  Scores  for  the  categories  are  plotted  directly  on  the  chart 
using  the  0-100  scale  on  the  side  of  the  chart.  Negative  Converted  Scores 
are  given  a  token  shaded  area  below  the  zero  line  and  the  actual  negative 
score  is  then  entered  in  the  category's  area  directly  below  the  shaded  portion. 
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CRITIQUE  PROCEDURE 

After  completing  the  numerical  evaluation  of  a  shelter  system  (ending 
with  completion  of  the  Operational  Capability  profile),  a  critique  should  be 
prepared  which  interprets  the  scores  received  on  each  category.  The  evaluator 
should  be  completely  familiar  with  the  item  weighting  and  scoring  system  used 
in  the  Evaluation  Instrument  before  attempting  this. 

Since  the  Critique  will  be  of  necessity  critical  in  nature,  concrete 
suggestions  as  to  how  the  shelter  system  can  be  improved  should  be  included 
whenever  possible.  The  form  and  extent  of  these  suggestions  are  of  course  a 
matter  of  policy  to  be  determined  by  the  Office  of  Civil  Defense. 

Several  points  to  be  kept  in  mind  when  preparing  a  critique  are  listed 
below  for  the  evaluator's  convenience. 

1.  No  system  failure  point  has  been  identified  on  the  positive 
portion  of  the  profile  scale.  However,  if  a  score  of  zero 
or  below  is  obtained  in  any  category  the  shelter  system  can 
be  considered  to  have  failed  in  accomplishing  its  objectives. 

2.  Whenever  a  category  receives  a  zero  or  negative  score,  the 
factor  or  factors  whose  omission  caused  the  situation  should 
be  identified  and  corrective  action  suggested. 

3.  The  seriousness  of  low  positive  scores  is  dependent  upon  the 
cause.  If  a  category  has  many  plan  factors  and  several  less 
important  factors  have  not  been  provided  for,  the  converted 
score  for  the  category  may  look  suspiciously  low.  Since 
certain  combinations  of  absent  factors  could  result  in  equal 
scores  but  unequal  total  effects,  inspection  of  individual 
items  is  also  recommended  for  low  positive  or  negative 
category  scores. 

4.  The  more  negative  a  score  the  more  deficient  the  system  is  in 
that  category. 


9 


